Surprise beyond prediction error
نویسندگان
چکیده
Surprise drives learning. Various neural "prediction error" signals are believed to underpin surprise-based reinforcement learning. Here, we report a surprise signal that reflects reinforcement learning but is neither un/signed reward prediction error (RPE) nor un/signed state prediction error (SPE). To exclude these alternatives, we measured surprise responses in the absence of RPE and accounted for a host of potential SPE confounds. This new surprise signal was evident in ventral striatum, primary sensory cortex, frontal poles, and amygdala. We interpret these findings via a normative model of surprise.
منابع مشابه
Surprise signals in the supplementary eye field: rectified prediction errors drive exploration-exploitation transitions.
Visual search is coordinated adaptively by monitoring and predicting the environment. The supplementary eye field (SEF) plays a role in oculomotor control and outcome evaluation. However, it is not clear whether the SEF is involved in adjusting behavioral modes based on preceding feedback. We hypothesized that the SEF drives exploration-exploitation transitions by generating "surprise signals" ...
متن کاملBeyond Reward Prediction Errors: Human Striatum Updates Rule Values During Learning.
Humans naturally group the world into coherent categories defined by membership rules. Rules can be learned implicitly by building stimulus-response associations using reinforcement learning or by using explicit reasoning. We tested if the striatum, in which activation reliably scales with reward prediction error, would track prediction errors in a task that required explicit rule generation. U...
متن کاملLearning to Expect Surprise: Hurricanes Harvey, Irma, Maria, and Beyond
Extreme events often bring unexpected situations and impacts, as the sequence of hurricanes and other natural disasters in summer and fall 2017 demonstrated. To reduce the risks associated with such events, many have focused on reducing uncertainty in prediction or reducing vulnerability. Although both are worthy goals, we suggest that the research community should also be focusing on the natur...
متن کاملSurprise and Curiosity for Big Data Robotics
This paper introduces a new perspective on curiosity and intrinsic motivation, viewed as the problem of generating behavior data for parallel off-policy learning. We provide 1) the first measure of surprise based on off-policy general value function learning progress, 2) the first investigation of reactive behavior control with parallel gradient temporal difference learning and function approxi...
متن کاملExpectation and surprise determine neural population responses in the ventral visual stream.
Visual cortex is traditionally viewed as a hierarchy of neural feature detectors, with neural population responses being driven by bottom-up stimulus features. Conversely, "predictive coding" models propose that each stage of the visual hierarchy harbors two computationally distinct classes of processing unit: representational units that encode the conditional probability of a stimulus and prov...
متن کامل